Attentive Experience Replay (AER)

1 Overview

Attentive Experience Replay (AER) uniformly samples \(\lambda\) times greater size than mini-batch size (\(k\)), then choose top \(k\) similar samples as minibatch.

The coefficient \(\lambda\) is annealed to 1 during training.

The similarity function \(\mathcal{F}(s_j,s_t)\) is task dependent. In the paper, the authors used cosine similarity for MuJoCo and norm of difference between embedded features for Atari 2600 games.

2 With cpprb

You can implement AER with simple ReplayBuffer class. Simply you can sample enlarged batch size then compute similarity.

3 References